Kolmogorov’s equations for jump Markov processes with unbounded jump rates

Countable State Markov Decision Processes with Unbounded Jump Rates and Discounted Cost: Optimality Equation and Approximations

This paper considers Markov decision processes (MDPs) with unbounded rates, as a function of state. We are especially interested in studying structural properties of optimal policies and the value function. A common method to derive such properties is by value iteration applied to the uniformised MDP. However, due to the unboundedness of the rates, uniformisation is not possible, and so value i...

Jump locations of jump-diffusion processes with state-dependent rates

We propose a general framework for studying statistics of jump-diffusion systems driven by both Brownian noise (diffusion) and a jump process with state-dependent intensity. Of particular natural interest in many physical systems are the jump locations: the system evaluated at the jump times. As an example, this could be the voltage at which a neuron fires, or the so-called ‘threshold voltage’....

On solutions of Kolmogorov’s equations for nonhomogeneous jump Markov processes

Article history: Received 23 February 2013 Available online 27 September 2013 Submitted by U. Stadtmueller

JUMP-Means: Small-Variance Asymptotics for Markov Jump Processes

Markov jump processes (MJPs) are used to model a wide range of phenomena from disease progression to RNA path folding. However, maximum likelihood estimation of parametric models leads to degenerate trajectories and inferential performance is poor in nonparametric models. We take a small-variance asymptotics (SVA) approach to overcome these limitations. We derive the small-variance asymptotics ...

Markov Chains and Jump Processes